Diffractive optical networks provide rich opportunities for visual computing tasks since the spatial information of a scene can be directly accessed by a diffractive processor without requiring any digital pre-processing steps. Here we present data class-specific transformations all-optically performed between the input and output fields-of-view (FOVs) of a diffractive network. The visual information of the objects is encoded into the amplitude (A), phase (P), or intensity (I) of the optical field at the input, which is all-optically processed by a data class-specific diffractive network. At the output, an image sensor-array directly measures the transformed patterns, all-optically encrypted using the transformation matrices pre-assigned to different data classes, i.e., a separate matrix for each data class. The original input images can be recovered by applying the correct decryption key (the inverse transformation) corresponding to the matching data class, while applying any other key will lead to loss of information. The class-specificity of these all-optical diffractive transformations creates opportunities where different keys can be distributed to different users; each user can only decode the acquired images of only one data class, serving multiple users in an all-optically encrypted manner. We numerically demonstrated all-optical class-specific transformations covering A-->A, I-->I, and P-->I transformations using various image datasets. We also experimentally validated the feasibility of this framework by fabricating a class-specific I-->I transformation diffractive network using two-photon polymerization and successfully tested it at 1550 nm wavelength. Data class-specific all-optical transformations provide a fast and energy-efficient method for image and data encryption, enhancing data security and privacy.

A unidirectional imager would only permit image formation along one direction, from an input field-of-view (FOV) A to an output FOV B, and in the reverse path, the image formation would be blocked. Here, we report the first demonstration of unidirectional imagers, presenting polarization-insensitive and broadband unidirectional imaging based on successive diffractive layers that are linear and isotropic. These diffractive layers are optimized using deep learning and consist of hundreds of thousands of diffractive phase features, which collectively modulate the incoming fields and project an intensity image of the input onto an output FOV, while blocking the image formation in the reverse direction. After their deep learning-based training, the resulting diffractive layers are fabricated to form a unidirectional imager. As a reciprocal device, the diffractive unidirectional imager has asymmetric mode processing capabilities in the forward and backward directions, where the optical modes from B to A are selectively guided/scattered to miss the output FOV, whereas for the forward direction such modal losses are minimized, yielding an ideal imaging system between the input and output FOVs. Although trained using monochromatic illumination, the diffractive unidirectional imager maintains its functionality over a large spectral band and works under broadband illumination. We experimentally validated this unidirectional imager using terahertz radiation, very well matching our numerical results. Using the same deep learning-based design strategy, we also created a wavelength-selective unidirectional imager, where two unidirectional imaging operations, in reverse directions, are multiplexed through different illumination wavelengths. Diffractive unidirectional imaging using structured materials will have numerous applications in e.g., security, defense, telecommunications and privacy protection.

随机且未知的散射介质背后的对象的分类为计算成像和机器视野字段的具有挑战性的任务。最新的基于深度学习的方法证明了使用图像传感器收集的扩散器延伸模式对对象进行分类。这些方法需要使用在数字计算机上运行的深神经网络进行相对大规模的计算。在这里，我们提出了一个全光处理器，使用单个像素检测到的宽带照明通过未知的随机相扩散器直接对未知对象进行分类。使用深度学习进行了优化的一组传播衍射层，形成了一个物理网络，该物理网络全面地绘制了随机扩散器后面输入对象的空间信息，以进入通过单个像素在输出平面上检测到的输出光的功率谱，衍射网络。我们在数值上使用宽带辐射通过随机新扩散器对未知手写数字进行分类，在训练阶段从未使用过，并实现了88.53％的盲目测试准确性。这种通过随机扩散器的单像素全光对象分类系统基于被动衍射层，该层可以通过简单地缩放与波长范围的衍射范围来缩放衍射特征，从而在电磁光谱的任何部分中运行，并且可以在电磁光谱的任何部分工作。这些结果在例如生物医学成像，安全性，机器人技术和自动驾驶中具有各种潜在的应用。

病理诊断依赖于组织学染色的薄组织样品的目视检查，其中使用不同类型的污渍来对比并突出各种所需的组织学特征。但是，破坏性的组织化学染色程序通常是不可逆的，因此很难在同一组织段上获得多个污渍。在这里，我们通过层叠的深神经网络（C-DNN）演示了虚拟的染色转移框架，以数字化将苏木精和曙红（H＆E）染色的组织图像转化为其他类型的组织学染色。与单个神经网络结构不同，该结构仅将一种染色类型作为一种输入来输出另一种染色类型的数字输出图像，C-DNN首先使用虚拟染色将自动荧光显微镜图像转换为H＆E，然后执行从H＆E到另一个域的染色转换以级联的方式染色。在训练阶段的这种级联结构使该模型可以直接利用H＆E和目标特殊污渍的组织化学染色图像数据。该优势减轻了配对数据获取的挑战，并提高了从H＆E到另一个污渍的虚拟污渍转移的图像质量和色彩准确性。我们使用肾针芯活检组织切片验证了这种C-DNN方法的出色性能，并将H＆E染色的组织图像成功地转移到虚拟PAS（周期性酸 - 雪）染色中。该方法使用现有的，组织化学染色的幻灯片提供了特殊污渍的高质量虚拟图像，并通过执行高度准确的污渍转换来创造数字病理学的新机会。

开发了基于深度学习的虚拟染色是为了将图像与无标签的组织截面形成鲜明对比，以数字方式与组织学染色相匹配，组织学染色是耗时，劳动密集型且与组织破坏性的。标准的虚拟染色需要在无标签组织的整个幻灯片成像过程中使用高的自动对焦精度，这会消耗总成像时间的很大一部分，并可能导致组织光损伤。在这里，我们介绍了一个快速的虚拟染色框架，该框架可以染色未标记的组织的散焦自动荧光图像，从而达到与无焦标签图像的虚拟染色相同的性能，还可以通过降低显微镜的自动焦点来节省大量的成像时间。该框架结合了一个虚拟自动化的神经网络，以数字重新聚焦了散落的图像，然后使用连续的网络将重新聚焦的图像转换为几乎染色的图像。这些级联网络构成了协作推理方案：虚拟染色模型通过培训期间的样式损失使虚拟自动化网络正常。为了证明该框架的功效，我们使用人肺组织训练并盲目地测试了这些网络。使用较低的焦点精度的4倍焦点，我们成功地将专注于重点的自动荧光图像转换为高质量的虚拟H＆E图像，与使用精心注重的自动荧光输入图像的标准虚拟染色框架相匹配。在不牺牲染色质量的情况下，该框架减少了无标签的全滑动图像（WSI）虚拟染色所需的总图像获取时间〜32％，同时降低了约89％的自动对焦时间，并且具有〜89％消除病理学中费力且昂贵的组织化学染色过程的潜力。

斑块测定是量化复制能力裂解病毒体浓度的黄金标准方法。加快和自动化病毒斑块分析将显着受益于临床诊断，疫苗开发以及重组蛋白或抗病毒药的产生。在这里，我们使用无透明全息成像和深度学习提出了快速且无染色的定量病毒斑块测定法。这种具有成本效益，紧凑和自动化的设备可显着减少传统斑块测定所需的孵化时间，同时保留其优于其他病毒定量方法的优势。该设备以每次测试井的对象捕获〜0.32 Giga像素/小时的相位信息，以无标签的方式覆盖约30x30 mm^2的面积，完全消除了染色。我们使用Vero E6细胞和囊泡气孔病毒证明了这种计算方法的成功。使用神经网络，此无染色装置最早在孵育后5小时内自动检测到第一个细胞裂解事件，并以100％的形式达到了> 90％的检测率（PFU）与传统的斑块测定法相比，特异性在<20小时内，可节省大量时间，而传统的牙菌斑测定时间约为48小时或更长时间。该数据驱动的牙菌斑测定还提供了量化细胞单层感染区域的能力，比标准病毒斑块分析的病毒浓度大10倍，对PFU和病毒感染区域进行自动计数和定量。这种紧凑，低成本的自动PFU定量设备可以广泛用于病毒学研究，疫苗开发和临床应用

人表皮生长因子受体2（HER2）生物标志物的免疫组织化学（IHC）染色在乳腺组织分析，临床前研究和诊断决策中广泛实践，指导癌症治疗和发病机制调查。 HER2染色需要由组织医学表演表演的艰苦组织处理和化学处理，这通常需要一天，以便在实验室中准备，增加分析时间和相关成本。在这里，我们描述了一种基于深度学习的虚拟HER2 IHC染色方法，其使用条件生成的对抗网络培训，训练以便将未标记/标记的乳房组织部分的自发荧光显微镜图像快速转化为明亮场当量的显微镜图像，匹配标准HER2 IHC染色在相同的组织部分上进行化学进行。通过定量分析证明了这一虚拟HER2染色框架的功效，其中三个董事会认证的乳房病理学家盲目地评级了HER2的几乎染色和免疫化化学染色的HER2整个幻灯片图像（WSIS），揭示了通过检查虚拟来确定的HER2分数IHC图像与其免疫组织化学染色的同类一样准确。通过相同的诊断师进行的第二种定量盲化研究进一步揭示了几乎染色的HER2图像在核细节，膜清晰度和染色伪像相对于其免疫组织化学染色的对应物的染色伪影等级具有相当的染色质量。这种虚拟HER2染色框架在实验室中绕过了昂贵，费力，耗时耗时的IHC染色程序，并且可以扩展到其他类型的生物标志物，以加速生命科学和生物医学工作流程的IHC组织染色。

Dynamic Graph Neural Networks (DGNNs) have been broadly applied in various real-life applications, such as link prediction and pandemic forecast, to capture both static structural information and temporal characteristics from dynamic graphs. Combining both time-dependent and -independent components, DGNNs manifest substantial parallel computation and data reuse potentials, but suffer from severe memory access inefficiency and data transfer overhead under the canonical one-graph-at-a-time training pattern. To tackle the challenges, we propose PiPAD, a $\underline{\textbf{Pi}}pelined$ and $\underline{\textbf{PA}}rallel$ $\underline{\textbf{D}}GNN$ training framework for the end-to-end performance optimization on GPUs. From both the algorithm and runtime level, PiPAD holistically reconstructs the overall training paradigm from the data organization to computation manner. Capable of processing multiple graph snapshots in parallel, PiPAD eliminates the unnecessary data transmission and alleviates memory access inefficiency to improve the overall performance. Our evaluation across various datasets shows PiPAD achieves $1.22\times$-$9.57\times$ speedup over the state-of-the-art DGNN frameworks on three representative models.

We propose a novel teacher-student model for semi-supervised multi-organ segmentation. In teacher-student model, data augmentation is usually adopted on unlabeled data to regularize the consistent training between teacher and student. We start from a key perspective that fixed relative locations and variable sizes of different organs can provide distribution information where a multi-organ CT scan is drawn. Thus, we treat the prior anatomy as a strong tool to guide the data augmentation and reduce the mismatch between labeled and unlabeled images for semi-supervised learning. More specifically, we propose a data augmentation strategy based on partition-and-recovery N$^3$ cubes cross- and within- labeled and unlabeled images. Our strategy encourages unlabeled images to learn organ semantics in relative locations from the labeled images (cross-branch) and enhances the learning ability for small organs (within-branch). For within-branch, we further propose to refine the quality of pseudo labels by blending the learned representations from small cubes to incorporate local attributes. Our method is termed as MagicNet, since it treats the CT volume as a magic-cube and $N^3$-cube partition-and-recovery process matches with the rule of playing a magic-cube. Extensive experiments on two public CT multi-organ datasets demonstrate the effectiveness of MagicNet, and noticeably outperforms state-of-the-art semi-supervised medical image segmentation approaches, with +7% DSC improvement on MACT dataset with 10% labeled images.

The task of referring video object segmentation aims to segment the object in the frames of a given video to which the referring expressions refer. Previous methods adopt multi-stage approach and design complex pipelines to obtain promising results. Recently, the end-to-end method based on Transformer has proved its superiority. In this work, we draw on the advantages of the above methods to provide a simple and effective pipeline for RVOS. Firstly, We improve the state-of-the-art one-stage method ReferFormer to obtain mask sequences that are strongly correlated with language descriptions. Secondly, based on a reliable and high-quality keyframe, we leverage the superior performance of video object segmentation model to further enhance the quality and temporal consistency of the mask results. Our single model reaches 70.3 J &F on the Referring Youtube-VOS validation set and 63.0 on the test set. After ensemble, we achieve 64.1 on the final leaderboard, ranking 1st place on CVPR2022 Referring Youtube-VOS challenge. Code will be available at https://github.com/Zhiweihhh/cvpr2022-rvos-challenge.git.